Repository navigation
feat(FRBC): correct a usage forecast no plan can absorb, instead of clipping it - #286
Conversation
FillRateBasedControlTUNES.send_leakage_behaviour read its fill-level scale from self._fill_level_scalefill_level_scale, an attribute name pasted in twice, which has been there since #158. The AttributeError that raised was caught by the handler's broad except as if the post had failed, so every FRBC.LeakageBehaviour was answered with PERMANENT_ERROR and none was ever recorded. mypy could not see it: the type-check task only looks at files that contain `from typing import`, and the FRBC base class is not one of them, so the attribute lookup went to Any. The next commit widens that. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
The type-check task fed mypy only the files that contain `from typing import`, which were seven of eighteen, and left out frbc_simple.py, both control-type __init__.py files and the example scripts. With --follow-imports skip, anything those files defined then looked like Any to the files that were checked, which is how the previous commit's misspelt attribute went unnoticed since #158. It now checks all of src, and with --check-untyped-defs, so the bodies of functions without annotations are checked too. What that surfaced, besides the attribute fixed in the previous commit: - get_access_token stored a token whose type could be None and then used it as a header value; it now returns the token, so the caller holds a str. - get_access_token and get_versions indexed a response typed as a dict or a list; each now says so plainly when the server answers with something other than an object, instead of failing on the lookup. - The S2 translations annotated their resolution as a string while frbc_simple passes a timedelta, which pandas accepts as readily; the annotation now says so. - FillRateBasedControlTUNES overrode trigger_schedule without the arguments the base class declares. It calls its own override without them and ignores them, so it now accepts and ignores them. - The example server indexed an account that can be None and spread a dict of mixed values into add_asset; it now says so when the demo user has no organisation, and types the dict. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
None of the S2 integration had a coverage number, for two reasons that each hid the other. The S2 tests run in a CI job of their own, since they need the s2 extra, and that job never asked for coverage. And .coveragerc omitted src/flexmeasures_client/s2/* outright, since #183, so asking for it would have reported nothing about S2 while looking as though it did. The omission is gone, and the S2 job now reports and uploads its own partial report, which Coveralls merges with the other job's in the finalize step, as it already does across the test matrix. The poe task that runs the S2 tests locally collects it as well. The job that skips tests/s2 will now list the S2 files as unexecuted in its own report; the merged figure is the one that counts. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
A CEM's handlers run for days with nobody watching standard output, so anything they print is lost to a deployment, which keeps the log instead. FRBCSimple gave up on a schedule that way when no storage status had arrived, which is the very failure, a storage left without an instruction, that the handler otherwise goes to some length to say out loud. It now logs a warning. A pygrep hook keeps it that way for s2/control_types and s2/cem.py. The example scripts print on purpose, as do the client's interactive delete confirmations, which answer an input() prompt on the console, so the hook leaves those alone. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
- The print ban now matches a print anywhere on a line and with a space before its parenthesis, plus pprint and writes to stdout or stderr, and covers all of s2/ but the example scripts. It only caught an indented `print(` at the start of a line before, in two directories, and missed s2/__init__.py, utils.py and wrapper.py, which run inside the CEM as much as the handlers do. - The example scripts get an __init__.py. Without one, coverage never listed websockets_server.py at all, since no test imports it, so the S2 report left it out instead of showing it at 0%. Running them by path is unaffected. - The TUNES leakage test now pins what is posted, its value and its unit, rather than only that something reached the sensor. - A client without a token now has a test that it authorises with the one it fetches. - FillRateBasedControlTUNES.trigger_schedule is back to taking no arguments, with the override mismatch marked for mypy instead. Accepting the base class's arguments and ignoring them would have turned a caller's mistake, such as expecting its superseded-request protection, into a silent one. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…s them in S2 lets the elements of an operation mode, and of a leakage behaviour, cover the fill-level range in pieces, and s2python checks that they are contiguous on a sorted copy, leaving the list in whatever order it was sent. Four places took an element by its position instead, and answered three different ways: - the measured power of an actuator status took elements[0], so a heat pump that derates as its buffer warms was recorded at its cold power when warm: 2000 W where it draws at most 1200 W; - the charging efficiency took elements[-1] as though it were the top of the range, so a mode listed high element first planned against the wrong ratio; - the storage efficiency took the leakage rate of elements[-1], on the same assumption, though the leakiest element need not be the top one; - the schedule translation selected by fill level, but settled a shared boundary by list order. element_for_fill_level now answers wherever a fill level is known: the element whose range holds it, the upper of two at a shared boundary, and past either end of the range the element that applies at that end, so the answer does not jump there. The measured power and the schedule translation use it. Before any storage status there is no fill level, so the measured power takes the bottom of the range and says in the log that it guessed. Where FlexMeasures takes one number for a whole schedule, it does not know which fill levels the schedule will pass through, so the most cautious element answers, which is the rule #253 already applies across modes: the lowest conversion efficiency, and the leakiest element. That is also independent of order. The efficiency now also skips an element that draws power while the storage does not fill, such as a standby or circulation mode. Before, such a mode gave a ratio of zero or below, which then won the minimum across modes, went onto the charging-efficiency sensor, and told the scheduler that no power could fill the storage at all; with the fill-level band priced rather than hard, never running the heat pump was then a feasible plan. Two guards were dropped rather than tested, because S2 makes them unreachable: every operation mode and leakage behaviour has at least one element, and every element at least one power range. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…ates Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…owest An adversarial review showed that the lowest efficiency over a mode's elements is not the cautious choice the previous commit called it. In FlexMeasures' linear program a charging efficiency scales what the actuator adds to the stock, so understating it protects the lower band only: the plan overshoots the upper band, buys more than it uses, and against a usage forecast the plan cannot relax, it can turn a problem the actuator can meet into one the scheduler finds infeasible. A heat pump at a COP of 4 cold and 2 warm, planned at 2, empties its planned buffer in a few hours while the real one keeps up. So within a mode, the element that applies at the fill level the schedule starts from answers. That is also the part of a schedule that runs before the next storage status asks for a new one, and it is what the actuator really does there. The storage efficiency likewise takes the leakage element at the latest fill level, and the leakiest only where no fill level is known yet. Across modes, the lowest still goes out, as #253 decided; the same argument applies to it, and that is left as a question rather than changed here. Also from the review: - A fill level that is not a number, which s2python lets through, counts as no fill level, and the selector refuses one, where it used to pick the bottom element silently. - The selector's docstring now says what is true: s2python checks contiguity for an operation mode only, never for a leakage behaviour; a single-point range, which the standard rules out, applies only at the top; and of two elements with the same range, the first listed answers. - Tests now cover the latest of several storage statuses, a fill-level scale other than 1 where scaling would pick another element, no warning for a mode with one element, and a schedule starting in the upper element. Each was a production mutation that passed the suite before. Formatted with black as well, which the previous commit was not: the pre-commit hooks were never installed in this repository, so they did not run on commit, and CI's check caught it. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
S2 makes an operation mode's diagnostic label optional, and FillRateBasedControlTUNES tells its modes apart by looking for "THP" or "NES" inside it. On a mode without one that raised a TypeError, so a single unlabelled mode stopped both planning and the recording of every mode's conversion efficiency. A missing label now simply matches neither. Found by type checking against s2python's own types, which also asked for two variables that held a range and then a number, or a message and then a series, to be given a name for each. New tests cover the planning path, the efficiency path, and posting a usage forecast, none of which any test reached before. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…inted request() ran check_content_type after its retry loop. It could never raise there: request_once parses every answer with aiohttp's json(), which refuses any content type but JSON, and the loop already reports that refusal as a ConnectionError, like every other failure to reach the API. Had it fired, it would have done harm. It built aiohttp's ContentTypeError from a message and a dict, where aiohttp expects the request's info and history, so anything that formatted the error, a log line included, raised an AttributeError of its own; and it read the body with response.text() without awaiting it. Its only test passed a mock whose text() was not a coroutine, and never formatted the error. The check is gone. The test that replaces it states where a non-JSON answer, such as a proxy's error page during a restart, is actually handled. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…ncluded mypy ran with --follow-imports skip, so s2python, pydantic, pandas and aiohttp were Any to it, and a misspelt attribute on an S2 message passed. It now follows imports, with pydantic's plugin, since S2 messages are pydantic models, and the pre-commit hook installs the s2 extra so those types are there to check. The plugin is also the guard against sliding back: without the extra it fails to load, rather than leaving every S2 type silently Any. A misspelt message_id on a leakage behaviour is now an error naming it. What it found, besides the two fixes in the previous commits: - get_validation_error_summary iterated a pydantic ValidationError, which is not iterable, so it raised a TypeError on any error it was given. Nothing called it, so it is gone rather than repaired. - Revocations were recorded by the RevokeObject's raw id, while every other id a handler keeps is text, so a lookup by the text of the id would have missed. They are recorded as text now. Nothing calls is_revoked yet, which means a revocation is acknowledged and then never consulted; that is recorded as a finding for later rather than changed here. - The rest is annotation: a message that arrives as a model, a dict or text is normalised to a dict once and checked as one; get_message returns the dict it always returned rather than claiming a str; the message type of a registered schema is read through model_fields rather than the deprecated __fields__; an operation mode's id is text where a label is wanted; and the actuator's power ranges are pairs rather than tuples of any length. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
… worked Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…ested An adversarial review ran every claim on this branch, and three were false: - The removed content-type check could fire. aiohttp's json() accepts JSON variants such as application/problem+json and application/vnd.api+json, and the check refused those, with an error that could not be printed. Removing it is right, but it changes behaviour: such an answer is now read like any other. The comment at the removed call, the test's docstring and the changelog entry now say so, instead of calling it dead code. - "A non-JSON answer is reported as a ConnectionError, as before" was only true of a wrong content type. A body that is not JSON under a JSON content type is refused by json() itself, as a JSONDecodeError, as it was before. - One unlabelled TUNES mode stopped planning, and the recording of the efficiencies of the modes listed after it, not of every mode. Two more overstatements live only in pushed commit messages and the PR body: following imports leaves pandas Any, since it ships no type information and no stubs are installed, and the efficiency path was already reached by a test. Tests now cover what five mutations showed was unpinned: a revocation taken by the CEM itself, with no control type active; a model of a type a handler does not handle; a usage forecast starting off the quarter hour; and one averaged within a quarter hour rather than taking its highest value. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…ions The schedule translation keeps a running fill level, so that each step is answered with the operation mode for the fill level the storage is then at. It advanced that level by a fill rate per second times a step in hours, so it moved 3600 times too little, and every step of a twelve-hour schedule was answered at the fill level the schedule started from (finding C1). Fixing that alone would have been worse: the level would then rise at the real rate and never fall, and pin every step at the top. So the integrator gets one written contract, the one S2 states for all three rates: a change in fill level per second, in the resource manager's own units, with usage and leakage lowering it where positive. Its step is a duration rather than a number of some unit. - Leakage now counts (C2). It read leakage_rate_per_second, which S2 does not have, and an actuator's leakage_behaviour, which cannot exist, both through getattr with a default, so the branch never ran. The handler now passes the latest FRBC.LeakageBehaviour, and the element that applies at the fill level. - Usage now counts. The handler passes the latest FRBC.UsageForecast, read per step by usage_rate_at; a frame may still carry a usage_forecast column instead, per second, which is what TUNES does, now dividing by the quarter hour it posts in. - The step is the schedule's own (B5): its duration over its number of values, which is the resolution of the sensor FlexMeasures answered at. It was taken to be a quarter hour whatever it was, so an hourly sensor's twelve values were laid on forty-eight quarter hours, pandas refused, and no instruction went out. - An instruction goes out wherever the mode or its factor changes, not only where the power does: at a constant power the fill level can pass from one element into another, which needs another factor. - The factor is the one that draws the power asked for (C4), taken from the power range, rather than worked back out of the fill rate, which failed wherever the fill rate was one value and the power a range: asked for 1200 W from a mode running 800 to 1200 W, that asked for 800 W. Where the power is one value and the fill rate a range, every factor draws that power, so the one that fills most is asked for. op_mode_compute_factor is gone. - The storage efficiency compounds a leakage rate per second (B10), where it compounded it per hour, declaring a buffer that loses about 7% over twelve hours as holding its heat to within 0.002%. - The handler's power and fill-rate measurements cover its own resolution (D9), rather than a quarter hour whatever it planned on. - translations.py is linted again: it opened with `# flake8: noqa`, which hid nothing, as it turns out. TUNES shares the translation, so its instructions change with it, for the better: it carried the same per-hour integration. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…showed it did not An adversarial review built its own integrator from S2's definitions, a second at a time, and found the previous commit agreed with it only while nothing changed within a step. Where something did, it did not: - A step that crossed from one element into another was integrated as if it had not: from 46, filling at 0.02 per second below 50 and 0.01 above, a quarter hour at factor 1 reaches 57, where holding the first element said 64, and at an hour's step the client thought a buffer full that was at 84. The same held for leakage. - Usage was sampled at the instant a step began, where FlexMeasures planned with the mean over the step, so a five-minute draw in each quarter hour counted fully or not at all, and a forecast starting off the step counted as nothing. follow_operation_mode now follows a step a minute at a time, as a resource manager follows an instruction: the mode and the factor are what the instruction fixes, while the element of the mode and of the leakage behaviour, and the usage a forecast expects, are each taken where and when they apply. The fill level is held between its bounds throughout, rather than only at the end of a step. Also from the review: - An instruction no longer goes out for a solver's own rounding: near factor 0, math.isclose without an absolute tolerance is exact equality, so 800 W give or take a ten-millionth from a mode running 800 to 1200 W sent 30 instructions in twelve hours instead of one. - A gap in a usage column, as FlexMeasures returns for a moment it holds nothing for, is no usage known, rather than a NaN that pinned the running fill level at the top of its bounds for the rest of the schedule. - FillRateBasedControlTUNES reads its usage forecast back divided by the fill-level scale it posted it with, which left it ten times too small at TUNES' default scale, and plans with its latest leakage behaviour. The previous commit message called TUNES' change "for the better"; without these it was not. - One power and a fill-rate range listed downwards asks for the top of the range, where operation_mode_conversion_efficiency declares the efficiency FlexMeasures plans with, so the instruction and the plan agree. - FRBCSimple refuses a schedule_duration that is not a whole number of resolution steps: FlexMeasures answers with whole steps only, so the step, read as the duration over the number of values, would come out wrong. - The handler's storage efficiency is what remains after one step of its own resolution, and FlexMeasures reads it per step of the leakage sensor's resolution, so the two have to agree. That is now said where the sensor is taken and where the value is posted. Tests cover each of these, and each of the four mutations the review found surviving, among them an on/off heat pump told to switch off: both modes ask for factor 0, and only the mode tells them apart. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…ed, and simulated The shipped example sent no instruction at all, for two reasons that only show with both halves running (findings A1, A2). Its resource manager sent its storage status before its system description, which planned nothing, since planning hangs off a storage status and needs a description to turn a schedule into instructions; and its description was dated 2026-01-15, while the server seeds prices from the day it starts, so a schedule's window had no prices. It also could not have shown steering if it had: one operation mode, with a power range of -0.5 to 0.5 W against a fill rate of -0.5 to 0.5 per second, which is 1.8 MW, over a declared range of 0.4. The resource manager is now a heat pump keeping a hot-water buffer warm. It is idle, or heats at anything from 400 to 1500 W, at a COP of 4 while the buffer is below 6 kWh and 3 above it. It declares its leakage, warmer losing more; a day of hot-water use with showers in the morning and washing up in the evening; and a comfort profile that pre-heats before the showers. It sends its description first, dated now, and then simulates its buffer from S2's own definitions, written separately from the CEM's code: it follows each instruction from the moment it is due, and reports its fill level every so often, which asks for a new schedule. That simulation is a second opinion on the CEM's arithmetic, and a test holds the two to agreement across an element boundary. The server gives the handler a sensor for every declaration (fill rate, band, usage, leakage, charging efficiency), so each can be seen next to the schedule planned against it, and plans from now rather than a day after the description. Its connection handling is split out of the aiohttp handler, so a test can serve the shipped resource manager with the shipped CEM, over an in-memory websocket, with only FlexMeasures stood in for. Two library changes the example needed: - A system description that arrives after the latest storage status, or that changes what the device can do, is planned for against that status (finding A3), rather than waiting for a status that a resource manager whose fill level holds steady may never send. It plans without re-posting the old status as a new measurement. - A schedule never starts before now. Planning happens on every storage status while a description can be days old, so a schedule anchored to the description alone slid into the past, and its instructions all arrived as one overdue burst. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…xMeasures FRBCSimple passed its energy_unit straight through as the flex model's soc-unit, while FlexMeasures validates soc-unit with OneOf(["kWh", "MWh"]). So four of the six units fill_rate_in_watts documents, J, kJ, MJ and Wh, made every trigger a 422, and the storage got no schedule at all (finding B2). The plan for this was to refuse those units when the handler is made. The carve source answers it better, by converting joules at the edge, and that is what happens now, for every supported unit: the state of charge and its two hard bounds go to FlexMeasures in kWh, converted from the recorded unit with the table fill_rate_in_watts already keeps, while kWh and MWh pass through as they are. The band, the usage and the leakage are read from sensors, which FlexMeasures converts itself, and the bounds the schedule is read back against stay in the recorded unit, since the operation modes are stated in it. An energy unit outside that table is now refused when the handler is made, where it used to surface only at planning time, as a logged failure to record a charging efficiency. The handler for that failure is gone, since nothing can now reach it. Not changed, and now pinned by a test: the measured state-of-charge sensor is never named in the flex model (finding B13 stands). Named there, FlexMeasures would save the state of charge it plans onto that sensor, beside the measurements. The carve source does name it, and the carve will have to leave that line behind. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…sures is sent Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…nges A second round of review showed that following a step a minute at a time traded one class of error for four others, and cost ten times the time: - Usage was sampled at the start of each minute. Half-minute draws starting on the half minute were never seen at all, and the same draws starting on the minute were counted twice. - A level settling on an element boundary, where filling stops above it, flipped across that boundary every step: 127 instructions in 144 steps, against one. - An element narrower than a minute's travel was skipped: a level that stops at 50 was carried to 80. - A 48-hour schedule against the largest forecast S2 allows took 1.7 s, on the event loop, scanning the forecast once a minute. Every rate is constant between the moments the usage changes and the moments the level reaches an element boundary, of the mode or of the leakage behaviour, or one of its bounds. So follow_operation_mode now steps from one such moment to the next and is exact. A level that reaches a boundary whose far side would push it back stays at that boundary, as a real storage does, and so does a level at one of its bounds, which rises again from there once filling wins. The usage forecast is read through a UsageProfile, built once per schedule and looked up by bisection. Also from the second round: - FillRateBasedControlTUNES posted its usage forecast as an amount per quarter hour labelled MW. FlexMeasures reads soc-usage as a power and multiplies it by the step's share of an hour, so it planned a quarter of the real draw. It now posts per hour, and reads back accordingly. - FRBCSimple refuses a resolution or schedule duration that is not positive, where a zero resolution raised a ZeroDivisionError. - The factor tolerance's justification was wrong: FlexMeasures rounds a schedule to six decimals of MW, so what it absorbs is the noise of converting to watts, not a solver's rounding. The tolerance stays; the comment and the changelog now say what it is for. - One of the first round's mutations had stopped applying after black split its line, so the gate's count overstated what was checked. The harness now rejects a mutant that cannot run, and every mutation is anchored to the formatted code. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…2-example-heat-pump Signed-off-by: F.N. Claessen <[email protected]>
…, and what review found An adversarial review of the example, and of the two library changes it needed, found the example resource manager following every plan it had ever been sent. Each instruction ran in a task of its own, nothing cancelled the tasks of a replaced batch, and the server sent no revocations. In the live demo against a sandboxed FlexMeasures, thirty instructions for the same quarter hour fired at once when it came, one from each replan. The server now sets send_revocations, so a newer plan revokes what is left of the one before, and the resource manager cancels a revoked instruction it has not yet followed, and acknowledges the revocation. Also from the review: - The leakage, the usage forecast and the comfort profile were sent once, at connect, and the storage efficiency was recorded once, for twelve hours from the message, so planning went without leakage after twelve hours and without usage or a band after a day. The resource manager now sends all three again every S2_EXAMPLE_FORECAST_INTERVAL seconds, from then on, and the server seeds its demo prices again every twelve hours. - In the library, a storage efficiency is now recorded each time a schedule is asked for, over the window it covers and as a share of the fill level it starts from, and named in the flex model only where it was recorded. On arrival, it covered the wrong hours, at a fill level nobody had reported yet, and as a share of the top of the range it understated the loss at lower fill levels. - align_to_resolution floored the local clock, which in the repeated hour of an autumn DST change took the first pass of that hour, so a schedule could start 67 minutes in the past. It now floors the instant, in UTC. - A state of charge is posted as a reading at an instant, where it was posted over the minute after it, which FlexMeasures stored as a forecast. - The example declares what it does: transitions between its two modes, which S2 lists as the ones possible; no PowerForecast, since it sends none; and a consumer of electricity rather than a storage of it. - Claims corrected: a description that differs from the one before it, its validity included, asks for a schedule, not only one that "changes what the device can do"; and a schedule starts in the current step, which has begun, not "never before now". - The test that held the example's buffer to the CEM's integrator compared two copies of one discretisation. The CEM is now exact and the buffer steps a minute at a time, so the test bounds their difference by a minute of the rate difference at the crossing, which is what two correct discretisations owe each other. Tests cover everything the review's thirteen mutations found unpinned. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
- The example now tells the CEM where each instruction stands: accepted on arrival, succeeded once carried out, revoked once revoked. Without that every instruction stayed NEW, so each replan revoked the whole previous batch, the instructions already carried out included. - The mode a transition reports leaving is the one the last carried-out instruction put the heat pump in. It was the mode the previous instruction to arrive asked for, which after a revocation the heat pump was never in. - A storage efficiency takes the loss as a share of the start fill level or of the middle of the storage's range, whichever is larger. A share of a near-empty start left almost nothing per step, and overstated the loss many times over once the plan filled the storage. - A state of charge is posted as an instant only on a sensor that records instants: FlexMeasures refuses a single value of no duration on a sensor with a resolution, so there it is posted over a minute, as before. - align_to_resolution floors on the moment's own clock, which is the grid FlexMeasures floors to, with the pass of an autumn DST hour taken from the moment. Flooring in UTC put hourly steps half an hour off in zones that are, and daily steps on the wrong midnight. - The example server's price reseeding survives a failed ingestion job, a wrong content type and a wrong API version, not only a lost connection. - Statuses about different steps that interleave within a second get belief times a second apart per step, where only the latest status was compared. The demo gate's first run had thirty statuses for one quarter hour arrive at once, and one was refused as a replacement. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
- In the repeated hour of an autumn DST change, a boundary is the later of the two candidates that is not after the moment. Taking the pass from the moment put 03:10 at two-hourly steps on 02:00 of the first pass, three hours back. The candidates are compared as pandas timestamps, which compare instants. - The changelog says what the floor does: on the clock of the moment, in the offset it carries, which is how FlexMeasures floors a posted series. "The site's own clock" was not true for series a resource manager anchors in UTC. - A state-of-charge sensor stating a resolution that is not a duration is posted over a minute and logged, where it aborted the status before planning. The docstring says the sensor is asked until it answers, which it is. - An instruction that has become due is no longer pending, so a revocation arriving while it is carried out is not answered as REVOKED for an instruction the heat pump is following. - The comment on the storage efficiency's reference says that below the middle of the range the loss is understated, and by how much at most. - Tests pin the forgetting of belief times more than a day old. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…er way The demo gate's second run, against a sandboxed FlexMeasures, heated the buffer in the quarter hour the plan chose, but every replan inside that quarter replanned the same quarter, at 0.72 kW, then 0.67, 0.63 and on down to 0.51 kW. A schedule asked for mid-step started at the boundary that step began at, while it was planned from the fill level reported now, which FlexMeasures takes to be the level at the schedule's start. So each replan counted on the whole step where only part of it was left, asked for less power over what remained, and the step delivered less than any plan meant it to. A schedule now starts at the next boundary, or at now where now is one. The step under way keeps the instruction it is already following, which a newer plan does not revoke, since it has been carried out. The end-to-end test stands the CEM's clock on a boundary, so that its schedules start there and the first instruction is due within the test. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…ix/frbc-energy-unit Signed-off-by: F.N. Claessen <[email protected]>
Black had split the expression around the trailing comment after the merge. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…cted request FlexMeasures merges the asset's stored flex context in at the top level of a request, and counts inflexible devices from there as well as from the electricity entry, to which the request also carries them. So each was counted twice: a house's planned aggregate power came out at twice its inflexible load, and a community planner saw a breach that was never realized (FlexMeasures/flexmeasures#2848). The request now names the same inflexible-device fields empty at the top level, which keeps FlexMeasures from merging the stored ones in there. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
S2 allows a negative expected usage, a gain. It was recorded as zero, which hid what the resource manager stated, to keep plans feasible without a usage correction. Recording and planning are separate questions: a schedule request now carries the latest forecast, without its negative values where no correction plans them, so the usage sensor can hold what was stated. A FlexMeasures server before v0.23.0 takes a usage only as a sensor, so there it is still recorded as zero. Until a forecast arrives, a handler plans without usage, since the sensor may hold negative usage recorded before it started. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…escription After a reconnection, a resource manager can send its actuator status before its system description. Only a description says what power an operation mode stands for, so such a status was logged and skipped, and the power sensor showed nothing until the next status. It is now kept, the latest per actuator, and recorded once a description arrives. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
… all operation modes As asked in the review of #283. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
S2 states leakage as a rate per fill-level range. It was planned as a storage efficiency, which FlexMeasures takes as a share of the stock, so the storage leaked in proportion to how full it was, exact at one fill level only. The leakage rate of the range the storage starts in is now added to the usage, over the whole window, and sent with the schedule request. With a usage correction, the correction covers usage and leakage together, so an empty storage's leakage can be taken away like any other forced usage. The leakage sensor records that rate, in the fill-rate unit, and the site sensor is now called "leakage". It needs FlexMeasures v0.23.0, which takes a usage with the request; an older server is planned without leakage. As asked in the review of #283. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
S2 states these rates per second, so in kWh/s a heat buffer's leakage of some tens of watts reads about 0.00001. They are now recorded in the power unit that goes with the energy unit (kW for kWh, W for J and Wh, MW for MJ and MWh), and the site's fill rate, usage forecast and leakage sensors are created in kW. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Like the energy and power units, the unit that rates of fill level are recorded in is the deployment's to set: W (the default), kW or MW. Watts read a residential heat buffer's leakage of some tens of watts as such; the site's fill rate, usage forecast and leakage sensors are created in W. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…nces Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
…ensor Signed-off-by: F.N. Claessen <[email protected]>
Every power and rate of fill level the CEM sets up is now in W, the handler's default: power, measured-power and usage correction joined fill rate, usage forecast and leakage. Watts read a residential site's small powers as such; a larger site passes its own sensors. An existing site keeps its sensors, and one in another unit than a new site would get is logged as a warning. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
FlexMeasures converts what is posted into a sensor's own unit, so an older site's sensor in kW is a valid choice, not a fault. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
|
Co-simulation results for this PR: two simulated homes with heat pumps, one day, steered as a community under a 2 kW limit. Each run uses deterministic solver settings.
Comfort was kept in every run, and no run fell back to its last plan. What this shows:
Next: find why the community steering converges to signals that leave the homes' plans over the limit when the correction is on. Until then, the correction (opt-in) should not be switched on for steered communities. |
S2 states fill rates and usage per second, which in kWh/s puts a heat buffer's rates near 1e-5. Ported from #289 without its leakage-as-usage work, which stays with #286. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Consistent with the handler's defaults, and readable at residential scale. Sensors an existing site already has are reused by name and keep their units. Ported from #289. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
|
The steering issue is resolved: the cause was in the co-simulation's community controller, not in this PR. Lifting the hold. Not a lack of rounds. In D7, every step reached a fixpoint, with no house re-planning, well inside the round cap. The controller read stale plans.
With the controller fixed (per slot, the latest belief across scheduler sources wins), the same run:
No fallbacks in D7c. This PR needs no change. Anything else that reads a site's schedule should resolve the latest belief across scheduler sources too, rather than rank sources. |
…ection #290 carried #289's unit changes on their own, so its conflicts resolve to this branch's side, which already has them along with the leakage as usage. #287's handling of an early actuator status and #288's changelog wording are kept, and the changelog links #290 next to #289. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
main holds the integration branch squashed (#279), with the same content, so this changes no file. It lets this PR target main. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW Signed-off-by: F.N. Claessen <[email protected]>
Base
main. The integration branch reached main through #279; this branch has it and main merged in, so the diff shows only #286 and #289. The unit changes of #289 reached main separately, through #290.This branch now also contains #289 (leakage planned as usage, and rates of fill level and site sensors in W), which was merged into it.
Why
S2 lets a resource manager expect any usage, negative included, and says nothing about what happens when the storage cannot take it. FlexMeasures plans a forecast usage as forced. So a gain at a full storage, or a draw at an empty one that the actuator cannot keep up with, leaves no feasible plan, and the storage gets no instruction. Until now the handler clipped negative usage to zero, which hid the first case and not the second.
What
With a
usage_correction_sensor_idand FlexMeasures v1.1.0 or newer,FRBCSimpleasks for a schedule with a correction of the usage forecast:A second device on the storage's stock (same state-of-charge sensor), on a commodity of its own, so it never touches the site's electricity. No instruction is ever sent for it.
Per step, it may only move the expected usage towards zero: give back up to the expected usage, or take away up to the expected gain. It works the same on both sides, since S2's fill levels do not say which bound is an overflow.
It is priced as a last resort: at twice an upper bound on what one MWh of stock can be worth to the scheduler. The request carries a coherent set of prices to make that bound finite and numerically sound:
soc_breach_price_factor);Device capacities are kept hard, so the correction cannot move a usage past zero.
A negative expected usage is then planned as stated. It is also recorded as stated without a correction (since 905254e, from a review comment on docs(changelog): say each S2 change plainly #283): there, the schedule request carries the forecast without its negative values. Only a FlexMeasures server before v0.23.0, which takes a usage only as a sensor, still gets it recorded as zero. The usage is sent inline, from the same forecast as the correction's limits and the translation back into instructions.
Where a correction cannot be planned (prices unreadable, a state-of-charge sensor with a resolution, a stored price that cannot be bounded, a stored flex context with commodities), the forecast is planned without its negative values, as before, with a warning saying why.
It is opt-in:
FRBCSiteConfigurator(correct_usage=True), orS2_CORRECT_USAGE=1for the example server. That is because it replaces FlexMeasures' default band breach price (1000 EUR/kWh) with a market-relative one, which is the deployment's call. Handlers without the new sensor behave exactly as before.How the price was chosen
Probes against FlexMeasures v1.1.0 measured, per scenario, the correction price above which the scheduler stops using a correction it does not need. That threshold is what a MWh of stock is worth to the scheduler. Each matched one term of the bound:
With FlexMeasures' default breach prices, a dominant price was about 1e8 EUR/MWh, where HiGHS already returned over-corrected plans. With the coherent set, over a 24-hour window with prices from −120 to 450 EUR/MWh, the correction price is about 1.8e6 EUR/MWh:
Priced at the band's breach price instead, the correction replaced band breaches (9.24 vs 3.12 kWh) and corrected where nothing needed correcting.
Why the site's capacity breach is left out. Pricing the correction above a site breach needs the request to set a site breach price the solver can live with. In the co-simulation (below), a site breach price of 100× the market price kept the correction solvable but let the houses ignore their steering capacity, and 1000× pushed the correction price to 1.3e7 EUR/MWh, where HiGHS declared every house schedule infeasible (replays put the cliff between 0.7e7 and 1.3e7 for this house). So a correction can now be cheaper than a site breach, while it stays dearer than a band breach, which the plan still uses first.
The asset's stored inflexible devices are carried into the electricity entry, and named empty at the top level, because FlexMeasures merges the stored context in there too and would count them twice (FlexMeasures/flexmeasures#2848).
FlexMeasures fills in its default site breach price only for a flex context without commodities; in a commodity entry, a site capacity without one stays hard. The request therefore states that default itself (FlexMeasures/flexmeasures#2846).
Evidence
relax-capacity-constraintswould soften the limits; and that efficiencies above one lowered the bound. All are fixed in the second commit, which also makes the correction opt-in.Co-simulation
One steered day, two houses, the community capacity policy, deterministic solver settings. Every run below had 0 degree-hours outside the comfort band.
🤖 Generated with Claude Code
https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW